WHAT IS CLAIMED IS: 



1 1 . A method of displaying multimedia information stored in a multimedia 

2 document on a display, the multimedia information comprising information of a plurality of 

3 types including information of a first type and information of a second type, the method 

4 comprising: 

5 displaying a graphical user interface (GUI) on the display; 

6 displaying, in a first area of the GUI, a representation of the multimedia 

7 information stored by the multimedia document, the displayed representation of the 

8 multimedia information comprising a representation of information of the first type and a 

9 representation of information of the second type; 

4 0 displaying a first lens covering a first portion of the first area; and 

g| 1 displaying, in a second area of the GUI, a representation of multimedia 

12 information displayed in the first portion of the first area, the representation of multimedia 

\3 information displayed in the second area comprising a portion of the representation of 

14 information of the first type covered by the first lens and a portion of the representation of 

1 5 information of the second type covered by the first lens. 

fij 1 2. The method of claim 1 wherein displaying the representation of the 

j 2 multimedia information stored by the multimedia document in the first area of the GUI 

W3 comprises: 

4 displaying a first thumbnail image in the first area of the GUI, the first 

5 thumbnail image comprising the representation of information of the first type; and 

6 displaying a second thumbnail image in the first area of the GUI, the second 

7 thumbnail image comprising the representation of information of the second type. 

1 3. The method of claim 1 wherein displaying, in the second area of the 

2 GUI, the representation of multimedia information displayed in the first portion of the first 

3 area comprises: 

4 displaying the portion of the representation of information of the first type 

5 covered by the first lens in a first panel in the second area of the GUI; and 

6 displaying the portion of the representation of information of the second type 

7 covered by the first lens in a second panel in the second area of the GUI. 
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1 4. The method of claim 1 wherein displaying, in the second area of the 

2 GUI, the representation of multimedia information displayed in the first portion of the first 

3 area comprises: 

4 determining a first time and a second time associated with the first lens; 

5 displaying, in the second area of the GUI, a representation of information of 

6 the first type occurring between the first time and the second time associated with the first 

7 lens; and 

8 displaying, in the second area of the GUI, a representation of information of 

9 the second type occurring between the first time and the second time associated with the first 
10 lens. 

Cji 5. The method of claim 1 further comprising: 

fg2 receiving user input moving the first lens to cover a second portion of the first 

[ J3 area; and 

TU4 responsive to the user input, automatically changing the information displayed 

' " 5 in the second area of the GUI such that the representation of multimedia information 

\ ~6 displayed in the second area of the GUI corresponds to the representation of multimedia 

HJ7 information included in the second portion of the first area. 

jjSl 6. The method of claim 1 further comprising: 

2 displaying a second lens covering a first portion of the second area; and 

3 displaying, in a third area of the GUI, a representation of multimedia 

4 information corresponding to the first portion of the second area, the representation of 

5 multimedia information displayed in the third area comprising a portion of the representation 

6 of information of the first type covered by the second lens and a portion of the representation 

7 of information of the second type covered by the second lens. 

1 7. The method of claim 6 wherein displaying, in the third area of the 

2 GUI, the representation of multimedia information corresponding to the first portion of the 

3 second area comprises: 

4 determining a first time and a second time associated with the second lens; 

5 displaying, in the third area of the GUI, a representation of information of the 

6 first type occurring between the first time and the second time associated with the second 

7 lens; and 
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8 displaying, in the third area of the GUI, a representation of information of the 

9 second type occurring between the first time and the second time associated with the second 
10 lens. 

1 8. The method of claim 6 wherein: 

2 displaying the representation of the multimedia information stored by the 

3 multimedia document in the first area of the GUI comprises: 

4 displaying a first thumbnail image in the first area of the GUI, the first 

5 thumbnail image comprising the representation of information of the first type; and 

6 displaying a second thumbnail image in the first area of the GUI, the 

7 second thumbnail image comprising the representation of information of the second type; 
H : 8 displaying the representation of multimedia information displayed in the first 
:" -9 portion of the first area in the second area of the GUI comprises: 

1 o displaying the portion of the representation of information of the first 

l#l type covered by the first lens in a first panel in the second area of the GUI; and 

ffl 

1-2 displaying the portion of the representation of information of the 

!.13 second type covered by the first lens in a second panel in the second area of the GUI; and 

14 displaying the representation of multimedia information corresponding to the 

fj5 first portion of the second area in the third area of the GUI comprises: 

1 6 displaying the representation of information of the first type 

17 corresponding to the first portion of the second area of the GUI in a first sub-area of the third 

18 area of the GUI; and 

1 9 displaying the representation of information of the second type 

20 corresponding to the first portion of the second area of the GUI in a second sub-area of the 

2 1 third area of the GUI. 

1 9. The method of claim 6 further comprising: 

2 receiving a user input moving the second lens to cover a second portion of the 

3 second area; and 

4 responsive to the user input, automatically changing the information displayed 

5 in the third area of the GUI such that the representation of multimedia information displayed 

6 in the third area of the GUI corresponds to the representation of the multimedia information 

7 included in the second portion of the second area. 

1 10. The method of claim 6 further comprising: 
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2 receiving a user input moving the first lens to cover a second portion of the 

3 first area; and 

4 responsive to the user input, automatically: 

5 changing the information displayed in the second area of the GUI such 

6 that the representation of multimedia information displayed in the second area of the GUI 

7 corresponds to the representation of multimedia information included in the second portion of 

8 the first area; and 

9 changing the information displayed in the third area of the GUI such 

1 0 that the representation of multimedia information displayed in the third area of the GUI 

1 1 corresponds to the representation of the multimedia information included in the second 
= 12 portion of the second area. 

b 

:{j li. The method of claim 6 further comprising: 

|*2 displaying a sub-lens covering a portion of the first area of the GUI 

3 corresponding to the first portion of the second area of the GUI covered by the second lens. 

* 1 12. The method of claim 1 1 further comprising: 

rU2 receiving a user input moving the second lens to cover a second portion of the 

'.' [ 3 second area; and 

O 4 responsive to the user input, automatically changing a position of the sub-lens 

* 5 to cover a portion of the first area of the GUI corresponding to the second portion of the 
6 second area. 

1 13. The method of claim 1 wherein: 

2 the information of the first type corresponds to video information; and 

3 the representation of the information of the first type comprises one or more 

4 video keyframes extracted from the video information. 

1 14. The method of claim 1 3 wherein: 

2 the information of the second type corresponds to audio information; and 

3 the representation of information of the second type comprises text 

4 information obtained from transcribing the audio information. 

1 15. The method of claim 1 3 wherein: 

2 the information of the second type corresponds to closed-caption (CC) text 

3 information; and 
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4 the representation of information of the second type comprises text 

5 information included in the CC text information. 

1 16. The method of claim 1 further comprising: 

2 receiving information indicating a user-specified concept of interest; and 

3 analyzing the multimedia information stored in the multimedia document to 

4 identify one or more locations in the multimedia information that are relevant to the user- 

5 specified concept of interest; 

6 wherein displaying the representation of multimedia information in the first 

7 area of the GUI comprises annotating the one or more locations in the multimedia 
u 8 information that are relevant to the user-specified concept of interest; and 

: --■ 9 wherein displaying, in the second area of the GUI, a representation of 

10 multimedia information displayed in the first portion of the first area comprises annotating 

11 the one or more locations in the multimedia information that are relevant to the user-specified 

12 concept of interest and that are located in the first portion of the first area. 

13 1 17. The method of claim 1 further comprising: 

:LH 2 receiving input indicating selection of a portion of the multimedia information 

M 3 occurring between a first time and a second time; and 

S -4 performing a first operation on the portion of the multimedia information 

5 occurring between a first time and a second time. 

1 1 8. A method of displaying multimedia information stored in a multimedia 

2 document on a display, the multimedia information comprising information of a first type and 

3 information of a second type, the method comprising: 

4 displaying a graphical user interface (GUI) on the display; 

5 displaying, in a first area of the GUI, a representation of the multimedia 

6 information stored by the multimedia document occurring between a start time (ts) and an end 

7 time (t e ) associated with the multimedia document, the displayed representation of the 

8 multimedia information comprising a representation of information of the first type occurring 

9 between t s and t e and a representation of information of the second type occurring between t s 

10 and t e , where (te > t s ); 

1 1 displaying a first lens emphasizing a portion of the first area of the GUI, the 

12 portion of the first area emphasized by the first lens comprising a representation of 
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1 3 multimedia information occurring between a first time (ti) and a second time (t 2 ), where (t s < 

14 ti < t 2 < t e ); and 

15 displaying, in a second area of the GUI, the representation of multimedia 

16 information occurring between ti and t 2 , the representation of multimedia information 

1 7 displayed in the second area comprising a representation of information of the first type 

1 8 occurring between ti and t 2 and a representation of information of the second type occurring 

19 between ti and t 2 . 

1 19. The method of claim 18 further comprising: 

2 displaying a second lens emphasizing a portion of the second area of the GUI, 

3 the portion of the second area emphasized by the second lens comprising a representation of 
t?4 multimedia information occurring between a third time ^3) and a fourth time (t 4 ), where (ti < 
P5 t 3 < t 4 < t 2 ); and 

£16 displaying, in a third area of the GUI, the representation of multimedia 

5^7 information occurring between t3 and U, the representation of multimedia information 

•||8 displayed in the third area comprising a representation of information of the first type 

9 occurring between t3 and U and a representation of information of the second type occurring 

f;iO between t 3 and U- 
L* 

pi 20. The method of claim 19 further comprising: 

"-2 changing the position of the first lens in response to user input such that the 

3 first lens emphasizes a portion of the first area of the GUI comprising a representation of 

4 multimedia information occurring between a fifth time (t 5 ) and a sixth time (t 6 ), where (t s < t 5 

5 < t 6 < t e ), (t 5 * ti), and (te * t 2 ); and 

6 responsive to the change in the position of the first lens, automatically 

7 displaying, in the second area of the GUI, the representation of multimedia information 

8 occurring between t 5 and t 6 , the representation of multimedia information displayed in the 

9 second area comprising a representation of information of the first type occurring between t 5 
and t 6 and a representation of information of the second type occurring between t 5 and U. 



10 



1 21. The method of claim 1 9 further comprising: 

2 changing the position of the second lens in response to user input such that the 

3 second lens emphasizes a portion of the second area of the GUI comprising a representation 
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4 of multimedia information occurring between a fifth time (t 5 ) and a sixth time (t 6 ), where (ti < 

5 t 5 <t6<t2),(t5*t 3 ),and(t 6 *t4);and 

6 responsive to the change in the position of the second lens, automatically 

7 displaying, in the third area of the GUI, the representation of multimedia information 

8 occurring between t 5 and t6, the representation of multimedia information displayed in the 

9 third area comprising a representation of information of the first type occurring between t 5 
1 0 and t 6 and a representation of information of the second type occurring between t 5 and t 6 . 

1 22. The method of claim 19 further comprising: 

2 displaying a third lens emphasizing a portion of the first area of the GUI 

3 comprising a representation of multimedia information occurring between t 3 and U. 

Ql 23 . The method of claim 22 further comprising: 

2 changing the position of the second lens in response to user input such that the 

H 3 second lens emphasizes a portion of the second area of the GUI comprising a representation 

i 4 of multimedia information occurring between a fifth time (t 5 ) and a sixth time (t 6 ), where (ti < 

5 t 5 < t 6 < t 2 ), (t 5 * t 3 ), and (t 6 * t 4 ); and 

6 responsive to the change in the position of the second lens, automatically 

pj7 changing the position of the third lens such that the third lens emphasizes a portion of the first 

;£J8 area of the GUI comprising a representation of multimedia information occurring between t 5 

fU9 and t 6 . 

1 24. The method of claim 18 wherein: 

2 the information of the first type is video information; 

3 the information of the second type is audio information; 

4 the representation of the information of the first type comprises one or more 

5 video keyframes extracted from the video information; and 

6 the representation of information of the second type comprises text 

7 information obtained from transcribing the audio information. 

1 25. The method of claim 18 wherein: 

2 the information of the first type is video information; 

3 the information of the second type is closed-caption (CC) text information; 

4 the representation of the information of the first type comprises one or more 

5 video keyframes extracted from the video information; and 
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6 the representation of the information of the second type comprises text 

7 information included in the CC text information. 

1 26. The method of claim 1 8 further comprising: 

2 receiving information indicating a first topic; and 

3 analyzing the multimedia information stored in the multimedia document to 

4 identify one or more locations in the multimedia information that are relevant to the first 

5 topic; 

6 wherein displaying the representation of the multimedia information stored by 

7 the multimedia document occurring between t s and t e in the first area of the GUI comprises 

8 highlighting the one or more locations in the multimedia information displayed in the first 
H9 area of the GUI; and 

10 wherein displaying the representation of multimedia information occurring 

1-1 between t t and t 2 in the second area of the GUI comprises highlighting the one or more 

Wl locations in the multimedia information that occur between times ti and t 2 . 

m 

7*1 27. The method of claim 18 further comprising: 

'~Z2 receiving input indicating selection of a portion of the multimedia information 

j flj3 occurring between a selection start time and a selection end time; and 
\ *4 performing a first operation on the portion of the multimedia information 

5 occurring between the selection start time and the selection end time. 

1 28. A method of displaying multimedia information stored in a multimedia 

2 document on a display, the multimedia information comprising video information and 

3 information of a first type, the method comprising: 

4 displaying a graphical user interface (GUI) on the display; 

5 displaying, in a first section of a first area of the GUI, a first set of one or more 

6 video keyframes extracted from the video information occurring between a start time (t s ) and 

7 an end time (t e ) associated with the multimedia document, where (te > t s ); 

8 displaying, in a second section of the first area of the GUI, text information 

9 corresponding to the information of the first type occurring between t s and te; 

10 displaying a first lens emphasizing a portion of the first section of the first area 

1 1 occurring between a first time (ti) and a second time (t 2 ) and a portion of the second section 

12 of the first area occurring between ti and t 2 , the emphasized portion of the first section of the 

1 3 first area comprising a second set of one or more video keyframes extracted from the video 
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14 information occurring between ti and t 2 , the emphasized portion of the second section of the 

1 5 first area comprising text information corresponding to information of the first type occurring 

16 between ti and t 2 , wherein the second set of one or more keyframes is a subset of the first set 

1 7 of one or more keyframes and (t s < ti < t 2 < t e ); 

1 8 displaying the second set of one or more keyframes in a first section of a 

1 9 second area of the GUI; and 

20 displaying text information corresponding to the information of the first type 

21 occurring between ti and t 2 in a second section of the second area of the GUI. 

1 29. The method of claim 28 further comprising: 

u 2 displaying a second lens emphasizing a portion of the first section of the 



3 second area and a portion of the second section of the second area, the emphasized portion of 

-4 the first section of the second area comprising a third set of one or more video keyframes 

5 extracted from the video information occurring between a third time (t 3 ) and a fourth time 

iu6 (t 4 ), the emphasized portion of the second section of the second area comprising text 

J3 

s 7 information corresponding to information of the first type occurring between t3 and U, 

" 8 wherein the third set of one or more video keyframes is a subset of the second set of one or 

" -9 more video keyframes and (ti < t 3 < t 4 < t 2 ); 



10 displaying a keyframe from the third set of one or more keyframes in a first 

; 1 1 section of a third area of the GUI; and 

12 displaying text information corresponding to the information of the first type 

13 occurring between t 3 and I4 in a second section of the third area of the GUI. 

1 30. The method of claim 28 further comprising: 

2 displaying a second lens emphasizing a portion of the first section of the 

3 second area and a portion of the second section of the second area, the emphasized portion of 

4 the first section of the second area comprising a third set of one or more video keyframes 

5 extracted from the video information occurring between a third time (t 3 ) and a fourth time 

6 (t 4 ), the emphasized portion of the second section of the second area comprising text 

7 information corresponding to information of the first type occurring between t 3 and t 4 , 

8 wherein the third set of one or more video keyframes is a subset of the second set of one or 

9 more video keyframes and (ti < t 3 < U ^ t 2 ); 

10 outputting video information starting from t 3 or from t 4 or from a time between 

11 t 3 and t 4 in a first section of a third area of the GUI; and 
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12 displaying text information corresponding to the information of the first type 

13 occurring between t3 and t4 in a second section of the third area of the GUI. 

1 31. The method of claim 28 wherein the information of the first type is 

2 audio information, and the text information corresponding to the information of the first type 

3 is obtained from transcribing the audio information. 

1 32. The method of claim 28 wherein the information of the first type is 

2 closed-caption (CC) text information, and the text information corresponding to the 

3 information of the first type is extracted from the CC text information. 

t =l 33. The method of claim 28 wherein the multimedia information stored by 

_;2 the multimedia document further comprises slides information, the method comprising: 
S3 displaying, in a third section of the first area of the GUI, a first set of one or 

: ,4 more slides extracted from the slides information occurring between t s and t e , wherein the 

1 -5 first lens emphasizes a portion of the third section of the first area comprising a second set of 

■il 

s 6 one or more slides extracted from the slides information occurring between ti and t 2 , the 

;=h7 second set of one or more slides is a subset of the first set of one or more slides; and 
J =8 displaying the second set of one or more slides in a third section of the second 

:9 area of the GUI 

1 34. The method of claim 33 further comprising: 

2 displaying a second lens emphasizing a portion of the first section of the 

3 second area, a portion of the second section of the second area, and a portion of the third 



4 section of the second area, the emphasized portion of the first section of the second area 

5 comprising a third set of one or more video keyframes extracted from the video information 

6 occurring between a third time (13) and a fourth time (14), the emphasized portion of the 

7 second section of the second area comprising text information corresponding to information 

8 of the first type occurring between t3 and U, the emphasized portion of the third section of the 

9 second area comprising a third set of one or more slides extracted from the slides information 
1 0 occurring between t3 and I4, wherein the third set of one or more video keyframes is a subset 



11 of the second set of one or more video keyframes, the third set of one or more slides is a 

12 subset of the second set of one or more slides, and (ti < t.3 < U < ti)', 

1 3 displaying at least one keyframe from the third set of one or more video 

14 keyframes in a first section of a third area of the GUI; 



78 



1 5 displaying text information corresponding to the information of the first type 

1 6 occurring between t3 and t 4 in a second section of the third area of the GUI; and 

1 7 displaying at least one slide from the third set of one or more slides in a third 

1 8 section of the third area of the GUI. 

1 35 . The method of claim 28 wherein the multimedia information stored by 

2 the multimedia document further comprises whiteboard images information, the method 

3 comprising: 

4 displaying, in a third section of the first area of the GUI, a first set of one or 



5 more whiteboard images extracted from the whiteboard images information occurring 

6 between t s and t e , wherein the first lens emphasizes a portion of the third section of the first 
ll area comprising a second set of one or more whiteboard images extracted from the 

08 whiteboard images information occurring between ti and X2, the second set of one or more 

jl9 whiteboard images is a subset of the first set of one or more whiteboard images; and 



10 displaying the second set of one or more whiteboard images in a third section 

H 1 of the second area of the GUI. 

JSl 1 36. The method of claim 3 5 further comprising : 

PJ2 displaying a second lens emphasizing a portion of the first section of the 

~ ;3 second area, a portion of the second section of the second area, and a portion of the third 

" "4 section of the second area, the emphasized portion of the first section of the second area 

5 comprising a third set of one or more video keyframes extracted from the video information 

6 occurring between a third time (t 3 ) and a fourth time (t 4 ), the emphasized portion of the 

7 second section of the second area comprising text information corresponding to information 

8 of the first type occurring between t 3 and t 4 , the emphasized portion of the third section of the 

9 second area comprising a third set of one or more whiteboard images extracted from the 

10 whiteboard images information occurring between t 3 and t 4 , wherein the third set of one or 

1 1 more video keyframes is a subset of the second set of one or more video keyframes, the third 

12 set of one or more whiteboard images is a subset of the second set of one or more whiteboard 

13 images, and (ti < t 3 < t 4 < t 2 ); 

14 displaying at least one keyframe from the third set of one or more video 

1 5 keyframes in a first section of a third area of the GUI; 

16 displaying text information corresponding to the information of the first type 

1 7 occurring between t 3 and U in a second section of the third area of the GUI; and 
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1 8 displaying a whiteboard images from the third set of one or more whiteboard 

1 9 images in a third section of the third area of the GUI. 

1 37. A system for displaying multimedia information stored in a multimedia 

2 document on a display, the multimedia information comprising information of a plurality of 

3 types including information of a first type and information of a second type, the system 

4 comprising: 

5 means for displaying a graphical user interface (GUI) on the display; 

6 means for displaying, in a first area of the GUI, a representation of the 

7 multimedia information stored by the multimedia document, the displayed representation of 
y& the multimedia information comprising a representation of information of the first type and a 
p.9 representation of information of the second type; 

TO means for displaying a first lens covering a first portion of the first area; and 

'Ul means for displaying, in a second area of the GUI, a representation of 

1 2 multimedia information displayed in the first portion of the first area, the representation of 

s 1 3 multimedia information displayed in the second area comprising a portion of the 

14 representation of information of the first type covered by the first lens and a portion of the 

1 1 5 representation of information of the second type covered by the first lens. 

b 

"1 38. A system for displaying multimedia information stored in a multimedia 

2 document on a display, the multimedia information comprising information of a first type and 

3 information of a second type, the system comprising: 

4 means for displaying a graphical user interface (GUI) on the display; 

5 means for displaying, in a first area of the GUI, a representation of the 

6 multimedia information stored by the multimedia document occurring between a start time 

7 (t s ) and an end time (te) associated with the multimedia document, the displayed 

8 representation of the multimedia information comprising a representation of information of 

9 the first type occurring between t s and t e and a representation of information of the second 

1 0 type occurring between ts and t e , where (t e > t s ); 

1 1 means for displaying a first lens emphasizing a portion of the first area of the 

12 GUI, the portion of the first area emphasized by the first lens comprising a representation of 

1 3 multimedia information occurring between a first time (ti) and a second time (t2), where (t s < 

14 ti < t 2 < t e ); and 
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15 means for displaying, in a second area of the GUI, the representation of 

16 multimedia information occurring between t t and t 2 , the representation of multimedia 

1 7 information displayed in the second area comprising a representation of information of the 

1 8 first type occurring between ti and t 2 and a representation of information of the second type 

19 occurring between ti and t 2 . 

1 39. A system for of displaying multimedia information stored in a 

2 multimedia document on a display, the multimedia information comprising video information 

3 and information of a first type, the system comprising: 

4 means for displaying a graphical user interface (GUI) on the display; 

5 means for displaying, in a first section of a first area of the GUI, a first set of 

6 one or more video keyframes extracted from the video information occurring between a start 
07 time (t s ) and an end time (te) associated with the multimedia document, where (t e > t s ); 

J8 means for displaying, in a second section of the first area of the GUI, text 

*9 information corresponding to the information of the first type occurring between ts and t e ; 

Jo means for displaying a first lens emphasizing a portion of the first section of 

rhl the first area occurring between a first time (ti) and a second time (t 2 ) and a portion of the 

!jf2 second section of the first area occurring between ti and t 2 , the emphasized portion of the first 

Ii3 section of the first area comprising a second set of one or more video keyframes extracted 

~p{4 from the video information occurring between ti and t 2 , the emphasized portion of the second 

1 5 section of the first area comprising text information corresponding to information of the first 

16 type occurring between ti and t 2 , wherein the second set of one or more keyframes is a subset 

1 7 of the first set of one or more keyframes and (t s < ti < t 2 < te); 

1 8 means for displaying the second set of one or more keyframes in a first section 

19 of a second area of the GUI; and 

20 means for displaying text information corresponding to the information of the 

21 first type occurring between ti and t 2 in a second section of the second area of the GUI. 

1 40. A computer program product stored on a computer-readable storage 

2 medium for displaying multimedia information stored in a multimedia document on a 

3 display, the multimedia information comprising information of a plurality of types including 

4 information of a first type and information of a second type, the computer program product 

5 comprising: 

6 code for displaying a graphical user interface (GUI) on the display; 
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7 code for displaying, in a first area of the GUI, a representation of the 

8 multimedia information stored by the multimedia document, the displayed representation of 

9 the multimedia information comprising a representation of information of the first type and a 

1 0 representation of information of the second type; 

1 1 code for displaying a first lens covering a first portion of the first area; and 

1 2 code for displaying, in a second area of the GUI, a representation of 



1 3 multimedia information displayed in the first portion of the first area, the representation of 

1 4 multimedia information displayed in the second area comprising a portion of the 

1 5 representation of information of the first type covered by the first lens and a portion of the 

1 6 representation of information of the second type covered by the first lens. 



u 1 41. The computer program product of claim 40 wherein the code for 

i 2 displaying the representation of the multimedia information stored by the multimedia 

093 document in the first area of the GUI comprises: 

[74 code for displaying a first thumbnail image in the first area of the GUI, the 

: : 5 first thumbnail image comprising the representation of information of the first type; and 

7 6 code for displaying a second thumbnail image in the first area of the GUI, the 

f, 7 second thumbnail image comprising the representation of information of the second type. 

h* 1 42. The computer program product of claim 40 wherein the code for 

S |2 displaying, in the second area of the GUI, the representation of multimedia information 

3 displayed in the first portion of the first area comprises: 

4 code for displaying the portion of the representation of information of the first 

5 type covered by the first lens in a first panel in the second area of the GUI; and 

6 code for displaying the portion of the representation of information of the 

7 second type covered by the first lens in a second panel in the second area of the GUI. 

1 43 . The computer program product of claim 40 wherein the code for 

2 displaying, in the second area of the GUI, the representation of multimedia information 

3 displayed in the first portion of the first area comprises: 

4 code for determining a first time and a second time associated with the first 

5 lens; 

6 code for displaying, in the second area of the GUI, a representation of 

7 information of the first type occurring between the first time and the second time associated 

8 with the first lens; and 
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9 code for displaying, in the second area of the GUI, a representation of 

10 information of the second type occurring between the first time and the second time 

1 1 associated with the first lens. 

1 44. The computer program product of claim 40 further comprising: 

2 code for receiving user input moving the first lens to cover a second portion of 

3 the first area; and 

4 code for responsive to the user input, automatically changing the information 

5 displayed in the second area of the GUI such that the representation of multimedia 

6 information displayed in the second area of the GUI corresponds to the representation of 

7 multimedia information included in the second portion of the first area. 

Hi 45 . The computer program product of claim 40 further comprising: 

p2 code for displaying a second lens covering a first portion of the second area; 

"3 and 

fc*4 code for displaying, in a third area of the GUI, a representation of multimedia 

■ ; g5 information corresponding to the first portion of the second area, the representation of 

..6 multimedia information displayed in the third area comprising a portion of the representation 

nil of information of the first type covered by the second lens and a portion of the representation 

^8 of information of the second type covered by the second lens, 
i ! 

fU 1 46. The computer program product of claim 45 wherein the code for 

2 displaying, in the third area of the GUI, the representation of multimedia information 

3 corresponding to the first portion of the second area comprises: 

4 code for determining a first time and a second time associated with the second 

5 lens; 

6 code for displaying, in the third area of the GUI, a representation of 

7 information of the first type occurring between the first time and the second time associated 

8 with the second lens; and 

9 code for displaying, in the third area of the GUI, a representation of 

10 information of the second type occurring between the first time and the second time 

1 1 associated with the second lens. 

1 47. The computer program product of claim 45 wherein: 
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2 the code for displaying the representation of the multimedia information stored 

3 by the multimedia document in the first area of the GUI comprises: 

4 code for displaying a first thumbnail image in the first area of the GUI, 

5 the first thumbnail image comprising the representation of information of the first type; and 

6 code for displaying a second thumbnail image in the first area of the 

7 GUI, the second thumbnail image comprising the representation of information of the second 

8 type; 

9 the code for displaying the representation of multimedia information displayed 

10 in the first portion of the first area in the second area of the GUI comprises: 

1 1 code for displaying the portion of the representation of information of 

12 the first type covered by the first lens in a first panel in the second area of the GUI; and 

Jj[3 code for displaying the portion of the representation of information of 

Q4 the second type covered by the first lens in a second panel in the second area of the GUI; and 
El 5 the code for displaying the representation of multimedia information 

56 corresponding to the first portion of the second area in the third area of the GUI comprises: 
417 code for displaying the representation of information of the first type 

I \ 8 corresponding to the first portion of the second area of the GUI in a first sub-area of the third 

19 area of the GUI; and 
hW code for displaying the representation of information of the second 

21 type corresponding to the first portion of the second area of the GUI in a second sub-area of 

22 the third area of the GUI. 

1 48. The computer program product of claim 45 further comprising: 

2 code for receiving a user input moving the second lens to cover a second 

3 portion of the second area; and 

4 responsive to the user input, code for automatically changing the information 

5 displayed in the third area of the GUI such that the representation of multimedia information 

6 displayed in the third area of the GUI corresponds to the representation of the multimedia 

7 information included in the second portion of the second area. 

1 49. The computer program product of claim 45 further comprising: 

2 code for receiving a user input moving the first lens to cover a second portion 

3 of the first area; and 

4 responsive to the user input, code for automatically: 
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5 changing the information displayed in the second area of the GUI such 

6 that the representation of multimedia information displayed in the second area of the GUI 

7 corresponds to the representation of multimedia information included in the second portion of 

8 the first area; and 

9 changing the information displayed in the third area of the GUI such 

1 0 that the representation of multimedia information displayed in the third area of the GUI 

1 1 corresponds to the representation of the multimedia information included in the second 

12 portion of the second area. 

1 50. The computer program product of claim 45 further comprising: 

2 code for displaying a sub-lens covering a portion of the first area of the GUI 
H; 3 corresponding to the first portion of the second area of the GUI covered by the second lens. 

ml 51. The computer program product of claim 50 further comprising: 

H2 code for receiving a user input moving the second lens to cover a second 

fU3 portion of the second area; and 

" 4 responsive to the user input, code for automatically changing a position of the 

; ' 5 sub-lens to cover a portion of the first area of the GUI corresponding to the second portion of 

PJ6 the second area. 

y 1 52. The computer program product of claim 40 wherein: 

2 the information of the first type corresponds to video information; and 

3 the representation of the information of the first type comprises one or more 

4 video keyframes extracted from the video information. 

1 53. The computer program product of claim 52 wherein: 

2 the information of the second type corresponds to audio information; and 

3 the representation of information of the second type comprises text 

4 information obtained from transcribing the audio information. 

1 54. The computer program product of claim 52 wherein: 

2 the information of the second type corresponds to closed-caption (CC) text 

3 information; and 

4 the representation of information of the second type comprises text 

5 information included in the CC text information. 
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1 55 . The computer program product of claim 40 further comprising: 

2 code for receiving information indicating a user-specified concept of interest; 

3 and 

4 code for analyzing the multimedia information stored in the multimedia 

5 document to identify one or more locations in the multimedia information that are relevant to 

6 the user-specified concept of interest; 

7 wherein the code for displaying the representation of multimedia information 

8 in the first area of the GUI comprises code for annotating the one or more locations in the 

9 multimedia information that are relevant to the user-specified concept of interest; and 

1 0 wherein the code for displaying, in the second area of the GUI, a 

1 1 representation of multimedia information displayed in the first portion of the first area 

CI2 comprises code for annotating the one or more locations in the multimedia information that 

3 3 are relevant to the user-specified concept of interest and that are located in the first portion of 

ft 4 the first area. 

;* 1 56. The computer program product of claim 40 further comprising: 

JU2 code for receiving input indicating selection of a portion of the multimedia 

PJ3 information occurring between a first time and a second time; and 
1^4 code for performing a first operation on the portion of the multimedia 

; r 5 information occurring between a first time and a second time. 

1 57. A computer program product stored on a computer-readable storage 

2 medium for displaying multimedia information stored in a multimedia document on a 

3 display, the multimedia information comprising information of a first type and information of 

4 a second type, the computer program product comprising: 

5 code for displaying a graphical user interface (GUI) on the display; 

6 code for displaying, in a first area of the GUI, a representation of the 

7 multimedia information stored by the multimedia document occurring between a start time 

8 (t s ) and an end time (te) associated with the multimedia document, the displayed 

9 representation of the multimedia information comprising a representation of information of 

1 0 the first type occurring between t s and te and a representation of information of the second 

1 1 type occurring between t s and t e , where (te > t s ); 

12 code for displaying a first lens emphasizing a portion of the first area of the 

1 3 GUI, the portion of the first area emphasized by the first lens comprising a representation of 
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14 multimedia information occurring between a first time (ti) and a second time (t 2 ), where (t s < 

15 ti < t 2 < tc); and 

1 6 code for displaying, in a second area of the GUI, the representation of 

1 7 multimedia information occurring between ti and t 2 , the representation of multimedia 

1 8 information displayed in the second area comprising a representation of information of the 

19 first type occurring between ti and t 2 and a representation of information of the second type 

20 occurring between ti and t 2 . 

1 58. The computer program product of claim 57 further comprising: 

2 code for displaying a second lens emphasizing a portion of the second area of 

3 the GUI, the portion of the second area emphasized by the second lens comprising a 

r -4 representation of multimedia information occurring between a third time (t 3 ) and a fourth 

55 time (t 4 ), where (ti < t 3 < t 4 < t 2 ); and 

f j 6 code for displaying, in a third area of the GUI, the representation of 

*7 multimedia information occurring between t 3 and t 4 , the representation of multimedia 

28 information displayed in the third area comprising a representation of information of the first 

1,9 type occurring between t 3 and U and a representation of information of the second type 

1 0 occurring between t 3 and t4. 

m 

5» J 1 59. The computer program product of claim 58 further comprising: 

flf 2 code for changing the position of the first lens in response to user input such 

3 that the first lens emphasizes a portion of the first area of the GUI comprising a 

4 representation of multimedia information occurring between a fifth time (t 5 ) and a sixth time 

5 (t 6 ), where (t s < t 5 < t 6 < t e ), (t 5 * ti), and (t 6 * t 2 ); and 

6 responsive to the change in the position of the first lens, code for automatically 

7 displaying, in the second area of the GUI, the representation of multimedia information 

8 occurring between t 5 and t6, the representation of multimedia information displayed in the 

9 second area comprising a representation of information of the first type occurring between t 5 
1 0 and t 6 and a representation of information of the second type occurring between t 5 and t 6 . 

1 60. The computer program product of claim 58 further comprising: 

2 code for changing the position of the second lens in response to user input 

3 such that the second lens emphasizes a portion of the second area of the GUT comprising a 
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4 representation of multimedia information occurring between a fifth time (t 5 ) and a sixth time 

5 (t 6 ), where (t t < t 5 < t 6 < t 2 ), (t 5 * t 3 ), and (t^ * t 4 ); and 

6 code for responsive to the change in the position of the second lens, 

7 automatically displaying, in the third area of the GUI, the representation of multimedia 

8 information occurring between t 5 and t 6 , the representation of multimedia information 

9 displayed in the third area comprising a representation of information of the first type 

1 0 occurring between t 5 and t6 and a representation of information of the second type occurring 

1 1 between t 5 and t 6 . 

1 61. The computer program product of claim 58 further comprising: 

2 code for displaying a third lens emphasizing a portion of the first area of the 
PJ3 GUI comprising a representation of multimedia information occurring between t 3 and t 4 . 

03 1 62. The computer program product of claim 61 further comprising: 

u2 code for changing the position of the second lens in response to user input 

:3 such that the second lens emphasizes a portion of the second area of the GUI comprising a 

s 4 representation of multimedia information occurring between a fifth time (t 5 ) and a sixth time 

j5 (t 6 ), where fa < t 5 < t 6 < t 2 ), (t 5 * t 3 ), and (t 6 * 14); and 

f 6 code for responsive to the change in the position of the second lens, 

Q 7 automatically changing the position of the third lens such that the third lens emphasizes a 

8 portion of the first area of the GUI comprising a representation of multimedia information 

9 occurring between t 5 and t 6 . 

1 63. The computer program product of claim 57 wherein: 

2 the information of the first type is video information; 

3 the information of the second type is audio information; 

4 the representation of the information of the first type comprises one or more 

5 video keyframes extracted from the video information; and 

6 the representation of information of the second type comprises text 

7 information obtained from transcribing the audio information. 

1 64. The computer program product of claim 57 wherein: 

2 the information of the first type is video information; 

3 the information of the second type is closed-caption (CC) text information; 
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4 the representation of the information of the first type comprises one or more 

5 video keyframes extracted from the video information; and 

6 the representation of the information of the second type comprises text 

7 information included in the CC text information. 

1 65. The computer program product of claim 57 further comprising: 

2 code for receiving information indicating a first topic; and 

3 code for analyzing the multimedia information stored in the multimedia 

4 document to identify one or more locations in the multimedia information that are relevant to 

5 the first topic; 

6 wherein the code for displaying the representation of the multimedia 

7 information stored by the multimedia document occurring between t s and t e in the first area of 
r|8 the GUI comprises code for highlighting the one or more locations in the multimedia 

19 information displayed in the first area of the GUI; and 

HO wherein the code for displaying the representation of multimedia information 

irl occurring between ti and t 2 in the second area of the GUI comprises code for highlighting the 

1 ! 2 one or more locations in the multimedia information that occur between times ti and t 2 . 

O 

r s .l 66. The computer program product of claim 57 further comprising: 

'2 code for receiving input indicating selection of a portion of the multimedia 

=3 information occurring between a selection start time and a selection end time; and 

4 code for performing a first operation on the portion of the multimedia 

5 information occurring between the selection start time and the selection end time. 

1 67. A computer program product stored on a computer-readable storage 

2 medium for displaying multimedia information stored in a multimedia document on a 

3 display, the multimedia information comprising video information and information of a first 

4 type, the computer program product comprising: 

5 code for displaying a graphical user interface (GUI) on the display; 

6 code for displaying, in a first section of a first area of the GUI, a first set of 

7 one or more video keyframes extracted from the video information occurring between a start 

8 time (t s ) and an end time (t e ) associated with the multimedia document, where (te > ts); 

9 code for displaying, in a second section of the first area of the GUI, text 

1 0 information corresponding to the information of the first type occurring between t s and t e ; 
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1 1 code for displaying a first lens emphasizing a portion of the first section of the 

12 first area occurring between a first time (ti) and a second time (t 2 ) and a portion of the second 

1 3 section of the first area occurring between ti and t 2 , the emphasized portion of the first section 

14 of the first area comprising a second set of one or more video keyframes extracted from the 

1 5 video information occurring between ti and t 2 , the emphasized portion of the second section 

1 6 of the first area comprising text information corresponding to information of the first type 

1 7 occurring between ti and t 2 , wherein the second set of one or more keyframes is a subset of 

1 8 the first set of one or more keyframes and (t s < ti < t 2 < te); 

1 9 code for displaying the second set of one or more keyframes in a first section 

20 of a second area of the GUI; and 

21 code for displaying text information corresponding to the information of the 

l£2 first type occurring between ti and t 2 in a second section of the second area of the GUI. 

□ 
£3 

1 68. The computer program product of claim 67 further comprising: 

f*2 code for displaying a second lens emphasizing a portion of the first section of 

r* 

IU3 the second area and a portion of the second section of the second area, the emphasized 

7'4 portion of the first section of the second area comprising a third set of one or more video 

|-^5 keyframes extracted from the video information occurring between a third time (t 3 ) and a 

fij6 fourth time (U), the emphasized portion of the second section of the second area comprising 

7 text information corresponding to information of the first type occurring between t 3 and t 4 , 

0J8 wherein the third set of one or more video keyframes is a subset of the second set of one or 

9 more video keyframes and (ti < t 3 < t 4 < t 2 ); 



1 0 code for displaying a keyframe from the third set of one or more keyframes in 

1 1 a first section of a third area of the GUI; and 

12 code for displaying text information corresponding to the information of the 

1 3 first type occurring between t 3 and U in a second section of the third area of the GUI. 

1 69. The computer program product of claim 67 further comprising: 

2 code for displaying a second lens emphasizing a portion of the first section of 

3 the second area and a portion of the second section of the second area, the emphasized 

4 portion of the first section of the second area comprising a third set of one or more video 

5 keyframes extracted from the video information occurring between a third time (t 3 ) and a 

6 fourth time (t4), the emphasized portion of the second section of the second area comprising 

7 text information corresponding to information of the first type occurring between t 3 and U, 
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8 wherein the third set of one or more video keyframes is a subset of the second set of one or 

9 more video keyframes and (ti < t3 < t4 < t2); 

1 0 code for outputting video information starting from t3 or from U or from a time 

1 1 between t 3 and U in a first section of a third area of the GUI; and 

12 code for displaying text information corresponding to the information of the 

1 3 first type occurring between t3 and t4 in a second section of the third area of the GUI. 

1 70. The computer program product of claim 67 wherein the information of 

2 the first type is audio information, and the text information corresponding to the information 

3 of the first type is obtained from transcribing the audio information. 

Ql 71. The computer program product of claim 67 wherein the information of 

ip the first type is closed-caption (CC) text information, and the text information corresponding 

*3 to the information of the first type is extracted from the CC text information. 

q 1 72. The computer program product of claim 67 wherein the multimedia 

\ J2 information stored by the multimedia document further comprises slides information, the 

w 

fU3 computer program product further comprising: 

code for displaying, in a third section of the first area of the GUI, a first set of 
one or more slides extracted from the slides information occurring between t s and t e , wherein 

6 the first lens emphasizes a portion of the third section of the first area comprising a second set 

7 of one or more slides extracted from the slides information occurring between ti and t 2 , the 

8 second set of one or more slides is a subset of the first set of one or more slides; and 

9 code for displaying the second set of one or more slides in a third section of 

1 0 the second area of the GUI 

1 73. The computer program product of claim 72 further comprising: 

2 code for displaying a second lens emphasizing a portion of the first section of 

3 the second area, a portion of the second section of the second area, and a portion of the third 



4 section of the second area, the emphasized portion of the first section of the second area 

5 comprising a third set of one or more video keyframes extracted from the video information 

6 occurring between a third time (t3) and a fourth time (U), the emphasized portion of the 

7 second section of the second area comprising text information corresponding to information 

8 of the first type occurring between t 3 and U, the emphasized portion of the third section of the 

9 second area comprising a third set of one or more slides extracted from the slides information 
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1 0 occurring between t 3 and t 4 , wherein the third set of one or more video keyframes is a subset 

1 1 of the second set of one or more video keyframes, the third set of one or more slides is a 

12 subset of the second set of one or more slides, and (ti < t 3 < t 4 < t2>; 

13 code for displaying at least one keyframe from the third set of one or more 

14 video keyframes in a first section of a third area of the GUI; 

1 5 code for displaying text information corresponding to the information of the 

16 first type occurring between t 3 and U in a second section of the third area of the GUI; and 

17 displaying at least one slide from the third set of one or more slides in a third 

1 8 section of the third area of the GUI. 

1 74. The computer program product of claim 67 wherein the multimedia 

H 2 information stored by the multimedia document further comprises whiteboard images 

- 3 information, the computer program product further comprising: 

* 4 code for displaying, in a third section of the first area of the GUI, a first set of 

jh* 5 one or more whiteboard images extracted from the whiteboard images information occurring 

%fi between t s and t e , wherein the first lens emphasizes a portion of the third section of the first 

~ 7 area comprising a second set of one or more whiteboard images extracted from the 

- 8 whiteboard images information occurring between ti and t2, the second set of one or more 
1 2 9 whiteboard images is a subset of the first set of one or more whiteboard images; and 

Q 0 code for displaying the second set of one or more whiteboard images in a third 

HI 

' 1 1 section of the second area of the GUI. 

1 75. The computer program product of claim 74 further comprising: 

2 code for displaying a second lens emphasizing a portion of the first section of 

3 the second area, a portion of the second section of the second area, and a portion of the third 

4 section of the second area, the emphasized portion of the first section of the second area 

5 comprising a third set of one or more video keyframes extracted from the video information 

6 occurring between a third time (t 3 ) and a fourth time (t 4 ), the emphasized portion of the 

7 second section of the second area comprising text information corresponding to information 

8 of the first type occurring between t 3 and t 4 , the emphasized portion of the third section of the 

9 second area comprising a third set of one or more whiteboard images extracted from the 

1 0 whiteboard images information occurring between t 3 and U, wherein the third set of one or 

1 1 more video keyframes is a subset of the second set of one or more video keyframes, the third 
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12 set of one or more whiteboard images is a subset of the second set of one or more whiteboard 

1 3 images, and (ti < t 3 < t 4 < t 2 ); 

14 code for displaying at least one keyframe from the third set of one or more 

1 5 video keyframes in a first section of a third area of the GUI; 

1 6 code for displaying text information corresponding to the information of the 

1 7 first type occurring between t 3 and U in a second section of the third area of the GUI; and 

1 8 code for displaying a whiteboard images from the third set of one or more 

1 9 whiteboard images in a third section of the third area of the GUI. 

1 76. A system for displaying multimedia information stored in a multimedia 

2 document, the multimedia information comprising information of a plurality of types 

t*'3 including information of a first type and information of a second type, the system comprising: 

P4 a display; 

v 

£^5 a processor; and 

H;6 a memory coupled to the processor, the memory configured to store a plurality 

TI of code modules for execution by the processor, the plurality of code modules comprising: 

' ,8 a code module for displaying a graphical user interface (GUI) on the 

■ 9 display; 

Llo a code module for displaying, in a first area of the GUI, a 

jp 1 representation of the multimedia information stored by the multimedia document, the 

1 2 displayed representation of the multimedia information comprising a representation of 

13 information of the first type and a representation of information of the second type; 

14 a code module for displaying a first lens covering a first portion of the 

15 first area; and 

16 a code module for displaying, in a second area of the GUI, a 

1 7 representation of multimedia information displayed in the first portion of the first area, the 

1 8 representation of multimedia information displayed in the second area comprising a portion 

19 of the representation of information of the first type covered by the first lens and a portion of 

20 the representation of information of the second type covered by the first lens. 

1 77. The system of claim 76 wherein the code module for displaying the 

2 representation of the multimedia information stored by the multimedia document in the first 

3 area of the GUI comprises: 
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4 a code module for displaying a first thumbnail image in the first area of the 

5 GUI, the first thumbnail image comprising the representation of information of the first type; 

6 and 

7 a code module for displaying a second thumbnail image in the first area of the 

8 GUI, the second thumbnail image comprising the representation of information of the second 

9 type. 

1 78. The system of claim 76 wherein the code module for displaying, in the 

2 second area of the GUI, the representation of multimedia information displayed in the first 

3 portion of the first area comprises: 

4 a code module for displaying the portion of the representation of information 
H c 5 of the first type covered by the first lens in a first panel in the second area of the GUI; and 
,56 a code module for displaying the portion of the representation of information 

■ 7 of the second type covered by the first lens in a second panel in the second area of the GUI. 

pj 1 79. The system of claim 76 wherein the code module for displaying, in the 

"2 second area of the GUI, the representation of multimedia information displayed in the first 

1 3 portion of the first area comprises: 

~ T 4 a code module for determining a first time and a second time associated with 

jjL~ 5 the first lens; 

!ftJ6 a code module for displaying, in the second area of the GUI, a representation 

7 of information of the first type occurring between the first time and the second time 

8 associated with the first lens; and 

9 a code module for displaying, in the second area of the GUI, a representation 

10 of information of the second type occurring between the first time and the second time 

1 1 associated with the first lens. 

1 80. The system of claim 76 wherein the plurality of code modules further 

2 comprises: 

3 a code module for receiving user input moving the first lens to cover a second 

4 portion of the first area; and 

5 responsive to the user input, a code module for automatically changing the 

6 information displayed in the second area of the GUI such that the representation of 

7 multimedia information displayed in the second area of the GUI corresponds to the 

8 representation of multimedia information included in the second portion of the first area. 
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1 81. The system of claim 76 wherein the plurality of code modules further 

2 comprises: 

3 a code module for displaying a second lens covering a first portion of the 

4 second area; and 

5 a code module for displaying, in a third area of the GUI, a representation of 

6 multimedia information corresponding to the first portion of the second area, the 

7 representation of multimedia information displayed in the third area comprising a portion of 

8 the representation of information of the first type covered by the second lens and a portion of 

9 the representation of information of the second type covered by the second lens. 

pi 82. The system of claim 81 wherein the code module for displaying, in the 

2 third area of the GUI, the representation of multimedia information corresponding to the first 

p3 portion of the second area comprises: 

if|j4 a code module for determining a first time and a second time associated with 

~5 the second lens; 

□6 a code module for displaying, in the third area of the GUI, a representation of 

ru 

7 information of the first type occurring between the first time and the second time associated 

:;.*8 with the second lens; and 

~ ; -9 a code module for displaying, in the third area of the GUI, a representation of 

1 0 information of the second type occurring between the first time and the second time 

1 1 associated with the second lens. 

1 83. The system of claim 81 wherein: 

2 the code module for displaying the representation of the multimedia 

3 information stored by the multimedia document in the first area of the GUI comprises: 

4 a code module for displaying a first thumbnail image in the first area of 

5 the GUI, the first thumbnail image comprising the representation of information of the first 

6 type; and 

7 a code module for displaying a second thumbnail image in the first 

8 area of the GUI, the second thumbnail image comprising the representation of information of 

9 the second type; 

10 the code module for displaying the representation of multimedia information 

1 1 displayed in the first portion of the first area in the second area of the GUI comprises: 
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12 a code module for displaying the portion of the representation of 

13 information of the first type covered by the first lens in a first panel in the second area of the 

14 GUI; and 

15 a code module for displaying the portion of the representation of 

16 information of the second type covered by the first lens in a second panel in the second area 

17 of the GUI; and 

1 8 the code module for displaying the representation of multimedia information 

19 corresponding to the first portion of the second area in the third area of the GUI comprises: 

20 a code module for displaying the representation of information of the 

21 first type corresponding to the first portion of the second area of the GUI in a first sub-area of 
Jj2 the third area of the GUI; and 

133 a code module for displaying the representation of information of the 

|24 second type corresponding to the first portion of the second area of the GUI in a second sub- 

3.5 area of the third area of the GUI. 

s 1 84. The system of claim 81 wherein the plurality of code modules further 

a 

■■'-2 comprises: 

■■ 3 a code module for receiving a user input moving the second lens to cover a 

1 14 second portion of the second area; and 

^5 responsive to the user input, a code module for automatically changing the 

6 information displayed in the third area of the GUI such that the representation of multimedia 

7 information displayed in the third area of the GUI corresponds to the representation of the 

8 multimedia information included in the second portion of the second area. 

1 85 . The system of claim 8 1 wherein the plurality of code modules further 

2 comprises: 

3 a code module for receiving a user input moving the first lens to cover a 

4 second portion of the first area; and 

5 responsive to the user input, a code module for automatically: 

6 changing the information displayed in the second area of the GUI such 

7 that the representation of multimedia information displayed in the second area of the GUI 

8 corresponds to the representation of multimedia information included in the second portion of 

9 the first area; and 
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1 0 changing the information displayed in the third area of the GUI such 

1 1 that the representation of multimedia information displayed in the third area of the GUI 

12 corresponds to the representation of the multimedia information included in the second 

13 portion of the second area. 

1 86. The system of claim 8 1 wherein the plurality of code modules further 

2 comprises: 

3 a code module for displaying a sub-lens covering a portion of the first area of 

4 the GUI corresponding to the first portion of the second area of the GUI covered by the 

5 second lens. 

0 i 87. The system of claim 86 wherein the plurality of code modules further 

P 

yG2 comprises: 

r73 a code module for receiving a user input moving the second lens to cover a 

" -4 second portion of the second area; and 

s 5 responsive to the user input, a code module for automatically changing a 

1 r 6 position of the sub-lens to cover a portion of the first area of the GUI corresponding to the 
I U 7 second portion of the second area. 

a 

p]l 88. The system of claim 76 wherein: 

2 the information of the first type corresponds to video information; and 

3 the representation of the information of the first type comprises one or more 

4 video keyframes extracted from the video information. 

1 89. The system of claim 88 wherein: 

2 the information of the second type corresponds to audio information; and 

3 the representation of information of the second type comprises text 

4 information obtained from transcribing the audio information. 

1 90. The system of claim 88 wherein: 

2 the information of the second type corresponds to closed-caption (CC) text 

3 information; and 

4 the representation of information of the second type comprises text 

5 information included in the CC text information. 
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1 91. The system of claim 76 wherein the plurality of code modules further 

2 comprises: 

3 a code module for receiving information indicating a user-specified concept of 

4 interest; and 

5 a code module for analyzing the multimedia information stored in the 

6 multimedia document to identify one or more locations in the multimedia information that are 

7 relevant to the user-specified concept of interest; 

8 wherein the code module for displaying the representation of multimedia 

9 information in the first area of the GUI comprises a code module for annotating the one or 
10 more locations in the multimedia information that are relevant to the user-specified concept 
H; l of interest; and 

12 wherein the code module for displaying, in the second area of the GUI, a 

f$3 representation of multimedia information displayed in the first portion of the first area 

i=f4 comprises a code module for annotating the one or more locations in the multimedia 

j|5 information that are relevant to the user-specified concept of interest and that are located in 

%6 the first portion of the first area. 

pjl 92. The system of claim 76 wherein the plurality of code modules further 

- 2 comprises: 

1 W3 a code module for receiving input indicating selection of a portion of the 

4 multimedia information occurring between a first time and a second time; and 

5 a code module for performing a first operation on the portion of the 

6 multimedia information occurring between a first time and a second time. 

1 93 . A system for displaying multimedia information stored in a multimedia 

2 document, the multimedia information comprising information of a first type and information 

3 of a second type, the system comprising: 

4 a display; 

5 a processor; and 

6 a memory coupled to the processor, the memory configured to store a plurality 

7 of code modules for execution by the processor, the plurality of code modules comprising: 

8 a code module for displaying a graphical user interface (GUI) on the 

9 display; 
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1 0 a code module for displaying, in a first area of the GUI, a 

1 1 representation of the multimedia information stored by the multimedia document occurring 

12 between a start time (t s ) and an end time (t e ) associated with the multimedia document, the 

13 displayed representation of the multimedia information comprising a representation of 

1 4 information of the first type occurring between t s and t e and a representation of information of 

1 5 the second type occurring between t s and t e , where (t e > t s ); 



16 a code module for displaying a first lens emphasizing a portion of the 

1 7 first area of the GUI, the portion of the first area emphasized by the first lens comprising a 

1 8 representation of multimedia information occurring between a first time (ti) and a second 

19 time (t 2 ), where (t s < ti < t 2 < t e ); and 

,, 20 a code module for displaying, in a second area of the GUI, the 

-21 representation of multimedia information occurring between ti and t 2 , the representation of 

f §2 multimedia information displayed in the second area comprising a representation of 

23 information of the first type occurring between ti and t 2 and a representation of information 

24 of the second type occurring between ti and t 2 . 

p 1 94. The system of claim 93 wherein the plurality of code modules further 

2 comprises: 

~H3 a code module for displaying a second lens emphasizing a portion of the 

■ 

~ 4 second area of the GUI, the portion of the second area emphasized by the second lens 

5 comprising a representation of multimedia information occurring between a third time (t 3 ) 

6 and a fourth time (U), where (ti < t 3 < U < t 2 ); and 

7 a code module for displaying, in a third area of the GUI, the representation of 

8 multimedia information occurring between t 3 and U, the representation of multimedia 

9 information displayed in the third area comprising a representation of information of the first 

10 type occurring between t 3 and t4 and a representation of information of the second type 

1 1 occurring between t 3 and t 4 . 

1 95. The system of claim 94 wherein the plurality of code modules further 

2 comprises: 

3 a code module for changing the position of the first lens in response to user 

4 input such that the first lens emphasizes a portion of the first area of the GUI comprising a 

5 representation of multimedia information occurring between a fifth time (ts) and a sixth time 

6 (te), where (t s < t 5 < t 6 < te), (t 5 * U), and (t 6 * t 2 ); and 
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7 responsive to the change in the position of the first lens, a code module for 

8 automatically displaying, in the second area of the GUI, the representation of multimedia 

9 information occurring between t 5 and t 6 , the representation of multimedia information 

10 displayed in the second area comprising a representation of information of the first type 

1 1 occurring between t 5 and t 6 and a representation of information of the second type occurring 

12 between t 5 and t 6 . 



1 96. The system of claim 94 wherein the plurality of code modules further 

2 comprises: 

3 a code module for changing the position of the second lens in response to user 

4 input such that the second lens emphasizes a portion of the second area of the GUI 

J^5 comprising a representation of multimedia information occurring between a fifth time (t 5 ) and 

:36 a sixth time (t 6 ), where (ti < t 5 < t 6 < t 2 ), (t 5 * t 3 ), and (t 6 * t 4 ); and 

27 responsive to the change in the position of the second lens, a code module for 



j?j8 automatically displaying, in the third area of the GUI, the representation of multimedia 

=09 information occurring between t 5 and t 6 , the representation of multimedia information 

10 displayed in the third area comprising a representation of information of the first type 

1 1 occurring between ts and U and a representation of information of the second type occurring 
1 12 between t 5 and t 6 . 



? " 1 97. The system of claim 94 wherein the plurality of code modules further 

2 comprises: 

3 a code module for displaying a third lens emphasizing a portion of the first 

4 area of the GUI comprising a representation of multimedia information occurring between t 3 

5 and t 4 . 

1 98. The system of claim 97 wherein the plurality of code modules further 

2 comprises: 

3 a code module for changing the position of the second lens in response to user 

4 input such that the second lens emphasizes a portion of the second area of the GUI 

5 comprising a representation of multimedia information occurring between a fifth time (t 5 ) and 

6 a sixth time (t 6 ), where (ti < t 5 < t 6 < t 2 ), (t 5 * t 3 ), and (t 6 * t 4 ); and 

7 responsive to the change in the position of the second lens, a code module for 

8 automatically changing the position of the third lens such that the third lens emphasizes a 
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9 portion of the first area of the GUI comprising a representation of multimedia information 
1 0 occurring between t 5 and t6. 

1 99. The system of claim 93 wherein: 

2 the information of the first type is video information; 

3 the information of the second type is audio information; 

4 the representation of the information of the first type comprises one or more 

5 video keyframes extracted from the video information; and 

6 the representation of information of the second type comprises text 

7 information obtained from transcribing the audio information. 

~ .\ 100. The system of claim 93 wherein: 

: ™ 2 the information of the first type is video information; 

j*3 the information of the second type is closed-caption (CC) text information; 

the representation of the information of the first type comprises one or more 

fl 5 video keyframes extracted from the video information; and 

*6 the representation of the information of the second type comprises text 

7 information included in the CC text information. 

0 1 101. The system of claim 93 wherein the plurality of code modules further 

* '2 comprises: 

3 a code module for receiving information indicating a first topic; and 

4 a code module for analyzing the multimedia information stored in the 

5 multimedia document to identify one or more locations in the multimedia information that are 

6 relevant to the first topic; 

7 wherein the code module for displaying the representation of the multimedia 

8 information stored by the multimedia document occurring between t s and t e in the first area of 

9 the GUI comprises a code module for highlighting the one or more locations in the 

1 0 multimedia information displayed in the first area of the GUI; and 

1 1 wherein the code module for displaying the representation of multimedia 

1 2 information occurring between ti and t2 in the second area of the GUI comprises a code 

1 3 module for highlighting the one or more locations in the multimedia information that occur 

14 between times ti and t2. 
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1 1 02. The system of claim 93 wherein the plurality of code modules further 

2 comprises: 

3 a code module for receiving input indicating selection of a portion of the 

4 multimedia information occurring between a selection start time and a selection end time; and 

5 a code module for performing a first operation on the portion of the 

6 multimedia information occurring between the selection start time and the selection end time. 

1 1 03. A system of displaying multimedia information stored in a multimedia 

2 document on a display, the multimedia information comprising video information and 

3 information of a first type, the system comprising: 

4 a display; 

fl5 a processor; and 

O 6 a memory coupled to the processor, the memory configured to store a 

L* 7 computer program; 

i u 8 wherein the processor is operative with the computer program to: 

-U 9 display a graphical user interface (GUI) on the display; 

]40 display, in a first section of a first area of the GUI, a first set of one or 

1 1 more video keyframes extracted from the video information occurring between a start time 

h\2 (t s ) and an end time (t e ) associated with the multimedia document, where (t e > t s ); 
Si 3 display, in a second section of the first area of the GUI, text 

14 information corresponding to the information of the first type occurring between t s and te; 

1 5 display a first lens emphasizing a portion of the first section of the first 

1 6 area occurring between a first time (ti) and a second time (t 2 ) and a portion of the second 

1 7 section of the first area occurring between ti and t 2 , the emphasized portion of the first section 

18 of the first area comprising a second set of one or more video keyframes extracted from the 

19 video information occurring between ti and t 2 , the emphasized portion of the second section 

20 of the first area comprising text information corresponding to information of the first type 

21 occurring between ti and t 2 , wherein the second set of one or more keyframes is a subset of 

22 the first set of one or more keyframes and (t s < ti < t 2 < t e ); 



23 display the second set of one or more keyframes in a first section of a 

24 second area of the GUI; and 

25 display text information corresponding to the information of the first 

26 type occurring between ti and t 2 in a second section of the second area of the GUI. 
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1 104. The system of claim 103 wherein the processor is operative with the 

2 computer program to: 

3 display a second lens emphasizing a portion of the first section of the second 

4 area and a portion of the second section of the second area, the emphasized portion of the first 

5 section of the second area comprising a third set of one or more video keyframes extracted 

6 from the video information occurring between a third time (t 3 ) and a fourth time (t4), the 

7 emphasized portion of the second section of the second area comprising text information 

8 corresponding to information of the first type occurring between t 3 and t 4 , wherein the third 

9 set of one or more video keyframes is a subset of the second set of one or more video 
1 0 keyframes and (ti < t 3 < U ^ t2); 

L l 1 display a keyframe from the third set of one or more keyframes in a first 

-1 2 section of a third area of the GUI; and 

1 3 display text information corresponding to the information of the first type 

1 4 occurring between t 3 and U in a second section of the third area of the GUI. 

€f 1 1 05 . The system of claim 1 03 wherein the processor is operative with the 

1 2 computer program to : 

3 display a second lens emphasizing a portion of the first section of the second 

h -- 4 area and a portion of the second section of the second area, the emphasized portion of the first 

5 section of the second area comprising a third set of one or more video keyframes extracted 

6 from the video information occurring between a third time (t3) and a fourth time (I4), the 

7 emphasized portion of the second section of the second area comprising text information 

8 corresponding to information of the first type occurring between t 3 and wherein the third 

9 set of one or more video keyframes is a subset of the second set of one or more video 

1 0 keyframes and (ti < t 3 < t4 < tz); 

1 1 output video information starting from t 3 or from U or from a time between t 3 

1 2 and t 4 in a first section of a third area of the GUI; and 

1 3 display text information corresponding to the information of the first type 

14 occurring between t 3 and U in a second section of the third area of the GUI. 

1 106. The system of claim 103 wherein the information of the first type is 

2 audio information, and the text information corresponding to the information of the first type 

3 is obtained from transcribing the audio information. 
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1 107. The system of claim 103 wherein the information of the first type is 

2 closed-caption (CC) text information, and the text information corresponding to the 

3 information of the first type is extracted from the CC text information. 

1 108. The system of claim 103 wherein the multimedia information stored by 

2 the multimedia document further comprises slides information, and wherein the processor is 

3 operative with the computer program to: 

4 display, in a third section of the first area of the GUI, a first set of one or more 

5 slides extracted from the slides information occurring between t s and t e , wherein the first lens 

6 emphasizes a portion of the third section of the first area comprising a second set of one or 

7 more slides extracted from the slides information occurring between ti and t 2 , the second set 
I* 8 of one or more slides is a subset of the first set of one or more slides; and 

: 9 display the second set of one or more slides in a third section of the second 

10 area of the GUI 

pj 1 1 09. The system of claim 108 wherein the processor is operative with the 

' ~ 2 computer program to: 

3 display a second lens emphasizing a portion of the first section of the second 

4 area, a portion of the second section of the second area, and a portion of the third section of 

5 the second area, the emphasized portion of the first section of the second area comprising a 

- 

-" : 6 third set of one or more video keyframes extracted from the video information occurring 

7 between a third time fo) and a fourth time (U), the emphasized portion of the second section 

8 of the second area comprising text information corresponding to information of the first type 

9 occurring between t 3 and t 4 , the emphasized portion of the third section of the second area 

10 comprising a third set of one or more slides extracted from the slides information occurring 

1 1 between t 3 and U, wherein the third set of one or more video keyframes is a subset of the 

12 second set of one or more video keyframes, the third set of one or more slides is a subset of 

1 3 the second set of one or more slides, and (ti < t 3 < t 4 < t 2 ); 

14 display at least one keyframe from the third set of one or more video 

1 5 keyframes in a first section of a third area of the GUI; 

16 display text information corresponding to the information of the first type 

1 7 occurring between t 3 and t 4 in a second section of the third area of the GUI; and 

1 8 display at least one slide from the third set of one or more slides in a third 

1 9 section of the third area of the GUI. 
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1 110. The system of claim 1 03 wherein the multimedia information stored by 

2 the multimedia document further comprises whiteboard images information, and wherein the 

3 processor is operative with the computer program to: 

4 display, in a third section of the first area of the GUI, a first set of one or more 



5 whiteboard images extracted from the whiteboard images information occurring between t s 

6 and te, wherein the first lens emphasizes a portion of the third section of the first area 

7 comprising a second set of one or more whiteboard images extracted from the whiteboard 

8 images information occurring between ti and t 2 , the second set of one or more whiteboard 

9 images is a subset of the first set of one or more whiteboard images; and 



10 display the second set of one or more whiteboard images in a third section of 

1 1 the second area of the GUI. 

q 1 111. The system of claim 1 1 0 wherein the processor is operative with the 

^ 2 computer program to : 

I* 3 display a second lens emphasizing a portion of the first section of the second 



JJ 4 area, a portion of the second section of the second area, and a portion of the third section of 

5 the second area, the emphasized portion of the first section of the second area comprising a 

; U 6 third set of one or more video keyframes extracted from the video information occurring 

7 between a third time (t 3 ) and a fourth time (t4), the emphasized portion of the second section 

= 8 of the second area comprising text information corresponding to information of the first type 

9 occurring between t 3 and t 4 , the emphasized portion of the third section of the second area 

1 0 comprising a third set of one or more whiteboard images extracted from the whiteboard 



1 1 images information occurring between t 3 and t 4 , wherein the third set of one or more video 

12 keyframes is a subset of the second set of one or more video keyframes, the third set of one or 

1 3 more whiteboard images is a subset of the second set of one or more whiteboard images, and 

14 (ti < t 3 < t 4 < t 2 ); 

1 5 display at least one keyframe from the third set of one or more video 

1 6 keyframes in a first section of a third area of the GUI; 

1 7 display text information corresponding to the information of the first type 

1 8 occurring between t 3 and U in a second section of the third area of the GUI; and 

1 9 display a whiteboard images from the third set of one or more whiteboard 

20 images in a third section of the third area of the GUI. 
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